Multiple paradigms in affective computing

This brief essay considers the three papers of the special issue of Interacting with Computers by Picard and colleagues, from several perspectives. First, I question two aspects of the work: the Computers Are Social Actors (CASA) approach, and the use of psychophysiological measurements of emotion without a stated theory of emotion. Despite these criticisms, the contributions of Picard and colleagues are valuable and powerfully challenging. I suggest three convergent ways of to pursue this important research program. I am grateful to Gilbert Cockton for the privilege and responsibility of commenting on the ground-breaking papers in Interacting with Computers by Rosalind Picard and associates (Klein, Moon, and Picard, 2002; Picard and Klein, 2002; Schreirer, Fernandez, Klein, and Picard, 2002). Although in many ways preliminary, their research calls us to reflect on the formal study of emotional interactions as conducted with and through computers, which until now have largely “reflect[ed] a view of the user as part of a productivity equation” (Picard and Klein, 2001). This is an original, provocative, and important challenge. As Cockton reminds us (2002), this work requires us to reconsider many of our theoretical and practical ideas about what emotion is, how it happens, and what we may want to do about it as we continue to do research and design with computers. In a similar vein – but with a very different methodology – Gaver and Martin (2000) have challenged what they call the “simple dichotomy between work and play” that can cause “[m]any devices [to] import values from the workplace into the home, emphasizing the requirement of ‘domestic work’ by allowing chores to be done more efficiently or productively.” McCarthy (2001) elaborates on some of these ideas in his argument to move from an HCI based in function to an HCI of “enchantment,” emphasizing delight, with overtones of magic. In another related development, Monk and colleagues have proposed alternatives to what Cockton has criticized (2002) as the “usability hardcore” approach to HCI, through a series of conferences called Computers and Fun (Computers and Fun, 1998, 2001). Together, these trends, critiques, and research programs challenge us to rethink our assumptions about the preconditions and limits of people’s experiences with computers, and with each other through computers. New ways of thinking may require new tools to § This is an expanded version of an invited article in Interacting with Computers, 16(4), 759-768 (August 2004). think with. In this commentary, I want to think about the diversity of approaches that HCI has used to think about the broader spectrum of human relationships to computers. What is it about Computers? Along with Reeves and Nass (e.g., 1996), Picard et al. focus on computers as social partners of humans. In this regard, I think it could be worthwhile to ask, what other social partners have humans related to? Which is to say, in what ways are computers uniquely social partners, as contrasted with other objects in people’s lives (i.e., the crucial claim of the “computers are social actors,” or CASA, argument)? 1 Picard (1997) considers the case of a non-living but animate wind-up toy dog that wags its tail, to which people may attribute emotions. The toy in question was a simulation of a living creature, with whom we might expect to have a social relationship. I want to examine the CASA argument with regard to a much broader set of non-living examples, going well beyond the inclusive “media” category of Reeves and Nass (1996). In western tradition, people often form strong affective attachments to boats (e.g., attributing female gender to them). People also form strong affective relationships with other moving objects, such as cars (there is at least one popular book and movie about an evil automobile – King, 1983; Columbia Tristar, 1983). Many people name their cars, just as many people name their boats. It is common to hear people talk about their dwelling places as entities with character and intent (e.g., “My house always knows when I’ve saved up some money, and then it invents a new repair that I have to spend it on”). Some people have similar relationships with sports equipment, such as (in the States) baseball bats – a type of personal attachment that may have historical echoes in (predominantly) men’s relationship with personal weapons, especially swords. More profoundly, we have also learned about human relationships to places and to the natural world (Berry, 1971/1972; Dillard, 1974; Williams, 2001). Perhaps computers are different from other objects and places, because we interact with computers. However, it seems clear that we may also have mixed-initiative relationships (i.e., interactions) with objects such as boats and cars, and arguably with simpler objects as well. For example, imagine writing with a familiar and comfortable pen whose ink supply has run low. Or, an example more closely related to the papers in Interacting with Computers, imagine typing at a keyboard in which a few keys require twice as much force as the other keys. Or, for contrasting cases, imagine playing music on a piano with perfect timbre, or rolling out a piecrust using a properly chilled marble rolling pin and board. Our affective responses in each of the situations are predictable and immediate. Berry (1971/1972) makes a powerful analogy between a man’s marriage to a woman and his relationship to his farm – a theme that he has pursued in fiction as well (most notably 1 In 1997, Picard described her research program (at that time) as not concerned with “how people feel about their computers, and how and why their feelings evolve as they do, even though these are important topics” (Picard, 1997). But five years later, the Reeves’ and Nass’s paradigm (which is concerned, in large part, precisely with how people feel about computers as social partners) has taken on a prominent position in the field of HCI, and has become an important part of Picard’s and colleagues’ affective computing approach. Schreirer et al. (2002) were concerned with identifying people’s responses to computers (or to computer behaviors, which are arguably isomorphic with computers themselves in the CASA paradigm), and Klein and Picard (2002) were concerned with how to reduce the frustration that people felt toward computers after such behaviors. It is therefore worthwhile to spend a little time discussing the controversial